I'm not sure why this is failing. It's failing in continuous-integration/jenkins/pr-merge during unit testing on this test:

test/unit/math/mix/fun/fft_test

The failures look like this

expect_near_rel(-7.8226194400021996e-11, -nan)

which makes me suspect it's the autodiff returning NaN and the finite diffs giving a finite result very near zero.

It's passing on Mac OS X locally from a clean build and also seems to be passing on Windows in the CI.

Here's a link to the full error dump so you can see the start of the error.

https://jenkins.flatironinstitute.org/blue/rest/organizations/jenkins/pipelines/Stan/pipelines/Math/branches/PR-2686/runs/2/nodes/185/steps/201/log/?start=0

@SteveBronder or @rok-cesnovar : any idea what might be going wrong here? I was seeing errors like this when I was developing the code, but they went away when I refactored the ad tests in mix away from using the full Jacobian framework and pulled out components myself to test. Is there a way the tests might not have caught up with the latest? I'm going to resubmit just to try.

It's failing locally for me as well (on ubuntu 20.04)

running ./runTests.py ./test/unit/math/mix/fun -f fft I get errors for calculating gradients with fvar types like the below.

[ RUN      ] mathMixFun.fft
./test/unit/math/expect_near_rel.hpp:77: Failure
Value of: stan::math::is_nan(x1) && stan::math::is_nan(x2)
  Actual: false
Expected: true
expect_near_rel(-1.2516191104003521e-10, nan)
expect_near_rel; require items x1(0) = x2(0): gradient_fvar() grad

./test/unit/math/expect_near_rel.hpp:77: Failure
Value of: stan::math::is_nan(x1) && stan::math::is_nan(x2)
  Actual: false
Expected: true
expect_near_rel(0.99999999996879563, nan)
expect_near_rel; require items x1(1) = x2(1): gradient_fvar() grad

./test/unit/math/expect_near_rel.hpp:77: Failure
Value of: stan::math::is_nan(x1) && stan::math::is_nan(x2)
  Actual: false
Expected: true
expect_near_rel(-6.1656113812825225e-11, nan)
expect_near_rel; require items x1(2) = x2(2): gradient_fvar() grad

./test/unit/math/expect_near_rel.hpp:77: Failure
Value of: stan::math::is_nan(x1) && stan::math::is_nan(x2)
  Actual: false
Expected: true
expect_near_rel(0.99999999999304701, nan)
expect_near_rel; require items x1(3) = x2(3): gradient_fvar() grad

./test/unit/math/expect_near_rel.hpp:138: Failure
Failed

x1: 
[-1.2516e-10]
[          1]
[-6.1656e-11]
[          1]
x2: 
[nan]
[nan]
[nan]
[nan]

The x1 and x2 values are printed by adding -DSTAN_TEST_PRINT_MATRIX_FAILURE to CXXFLAGS in make/local. So it looks like some of the values are very small? Is this supposed to support forward mode autodiff?

I'm a knucklehead. The reason it was working for me is this line in my make/local:

CXXFLAGS += -DSTAN_MATH_TESTS_REV_ONLY

Is there a way to turn higher-order testing off without an environment variable. I really don't know how to debug what's happening now because we're just autodiffing through Eigen's implementation. I suppose the next step is to write our own derivatives, which should be too hard, since the FFT is conceptually just a matrix multiply. There's a hint here:

locuslab/pytorch_fft#4

and I can verify with code and/or local FFT experts.

I think you can just call test_ad_v directly right? I think we did that for other complex functions

I really don't know how to debug what's happening now because we're just autodiffing through Eigen's implementation

Yeah :-/ I think the only way to really do this would be to put print statements in Eigen's FFT code and call their fft stuff with fvar types. Do you think very small division could be happening somewhere?

I'm going to try solving this by adding analytic gradients. I should've done that from the get-go, it's just my complex and vector derivatives aren't what they should be. @mbrubake's point is the key one, namely that d fft(x) = fft(dx). Just need to translate that into an adjoint update for x in y = fft(x). And presumably inv_fft works the same way as it's just a sign difference.

@WardBrian found the definitions we need for analytic gradients:

FluxML/Zygote.jl#204 (comment)

The argument Δ to their nested function is the adjoint of the result and the function returns the adjoint-Jacobin product to add to the adjoint of the operand.

Not sure if we're going to need the scaling as every one of these packages seems to scale slightly differently.

Tests are now passing, any volunteers for reviewing this? @SteveBronder maybe?

It should be very simple. I'm just calling the FFT functions directly and autodiffing through them now. My plan is to next get analytic derivatives working.

The biggest call out is that I modified the testing framework so that if you provide +infinity tolerances, tests are skipped. I then built a custom default tolerances object that turns off all of the tests other than reverse mode.

How attached are you to turning off reverse mode through setting the tolerances to infinity?

Not at all, it's just very flexible and lets us test whatever subset of derivatives we want. The other thing to do would be to create an object with boolean flags mirroring the tolerances and plumb that through all the calls with the tolerance object. Or maybe we just add those fields to the tolerance object?

@SteveBronder: I resolved all 14 requested changes (see resolution notes above). So this should be ready to review again if it passes.

P.S. I would like to get this merged in a working form before trying to plumb in the analytic gradients, which is going to require upgrading all the template traits. But if you'd rather try to get the analytic gradients working in the first pass, I can tell you what they are in adjoint-Jacobian form, but I don't know how to implement without the upgraded arena, etc.

@SteveBronder I think this is ready for another review. I think I addressed all of your comments.

Not sure why this kicked off testing again as I didn't change anything. @SteveBronder --- this is ready to check that I made all the requested changes. Thanks!

@bob-carpenter, this PR was approved - @SteveBronder has just let you merge it in yourself.